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FOREWORD 


Selection  and  classification  testing  throughout  the  nation  has  been 
subject  to  widespread  criticism  and  legal  attack.  Tests  used  by  the  military 
service,  however,  have  been  largely  immune  from,  the  criticism.  A  major  reason 
is  that  from  their  inception,  military  tests  have  been  carefully  validated  as 
predictors  of  successful  performance.  The  traditional  criterion  measure  in 
the  military  has  been  success  in  skill  training  courses. 

In  this  report  a  significant  new  measure  of  successful  performance  has 
been  used  as  the  criterion  measure.  Job  proficiency  tests,  developed  by  the 
Army  to  assess  performance  on  critical  job  tasks  and  identify  training 
deficiencies,  were  used  as  the  criterion  for  evaluating  the  effectivenss  of 
the  ASVAR  and  for  developing  new  aptitude  composites.  The  results  show  that 
ASVAB  is  an  effective  predictor  of  job  proficiency. 

This  research  was  done  by  the  Personnel  Utilization  Technical  Area  in 
response  to  requirements  of  Army  Project  2Q763731A791 . 

(  JOSEPH  ZmpiJER 
\Tedh  n  i  c  a  1 Vi  r  ec  t  o  r 
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APTITUDF  COMPOSITES  FOR  ASVAB  8,  9,  and  10 


BRIEF 


Requirements : 

Revised  aptitude  composites  were  required  for  new  forms  of  the  ASVAB  (8/9/10) 
implemented  on  1  October  1980.  Validation  of  the  composites  was  accomplished 
bv  determining  the  most  valid  predictors  of  Lraining  success  and  job  profi- 
c iencv . 

Procedures : 

Skill  Qualification  Tests  (SQT)  and  training  performance  scores  were  used  as 
the  criterion  measures  of  job  proficiency.  The  most  valid  sets  of  ASVAB  sub¬ 
tests  were  selected  for  the  aptitude  composites. 


Results : 

The  ASVAB  aptitude  composites  had  validity  coefficients  ranging  from  .52  to 
.75  for  predicting  training  success  and  job  proficiency  measured  from 
several  months  to  three  years  after  the  ASVAB  was  administered .  The  validity 
of  the  composites  is  adequate  to  justify  their  operational  use  for  selection 
and  classification  of  recruits.  The  results  support  the  usefulness  of  ASVAB 
as  a  valid  predictor  of  proficiency  and  the  usefulness  of  SOTs  as  measures  of 
job  proficiency. 

Util  izat ion 

"the  aptitude  composites  were  implemented  on  1  October  1980,  along  with  the 
new  forms  of  the  ASVAB,  for  use  as  screens  to  help  determine  qualification  of 
applicants  for  enlistment  and  for  assignment  of  recruits  to  skill  training 
programs . 
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APTITUDE  COMPOSITES  FOR  ASVAB  8,  9,  and  10 


PURPOSE  OF  THE  ASVAB 


The  Armed  Services  Vocational  Aptitude  Battery  (ASVAB)  is  used  by  all  ser¬ 
vices  to  determine  mental  qualification  for  enlistment  and  to  classify  accessions 
into  skill  training  programs.  The  ASVAB  provides  an  Armed  Forces  Qualification 
Test  (AFQT)  score,  which  is  used  to  screen  out  applicants  unqualified  for 
enlistment.  The  ASVAB  also  provides  aptitude  composites,  which  serve  a  dual 
purpose:  one  is  as  supplementary  enlistment  standards  to  help  determine 
qualification  of  those  applicants  who  pass  the  AFQT;  the  second  purpose  is  to 
determine  eligibility  for  assignment  to  job  specialties.  The  ASVAB  was  imple¬ 
mented  in  January  1976  as  the  replacement  for  the  test  batteries  used  until 
that  time  by  the  services. 

PROBLEM 


New  forms  of  the  ASVAB  have  been  developed  to  replace  the  original  inter - 
service  battery  (forms  6  and  7).  The  original  version  contained  13  subtests, 
while  the  replacement  forms  (8/9/10)  contain  only  10  subtests.  Changes  in  the 
content  of  the  ASVAB  also  necessitate  changes  in  the  aptitude  composites. 

The  subtests  in  both  the  current  and  replacement  versions,  together  with  a 
brief  description  of  each,  are  listed  in  Table  1.  Each  service  has  developed 
its  own  set  of  composites,  based  on  its  job  structure  and  the  validity  of  the 
subtests  for  its  training  programs. 

The  Army  has  had  nine  aptitude  composites  since  1973.  Each  composite  is 
used  as  a  prerequisite  to  determine  qualification  for  a  set  of  related  skill 
training  programs.  Successful  completion  of  the  training  program  results  in 
the  award  of  a  Military  Occupational  Specialty  (MOS).  For  example  one  apti¬ 
tude  composite  is  labeled  CO  for  Combat,  and  is  used  to  classify  recruits  into 
infantry  and  armor  specialties;  another  composite  is  labeled  EL,  for  Elec¬ 
trodes  Repair,  and  is  used  for  all  electronic  repair  specialties  in  the 
signal  and  air  defense  fields.  Each  of  the  original  nine  composites  contained 
three  to  five  subtests  that  were  found  to  be  the  most  va’id  predictors  of 
success  in  the  job  training  programs.''-  The  composites  were  developed  for 
tne  Army  Classification  Battery,  the  predecessor  to  the  ASVAB  in  tae  Army. 
Since  the  Army  Classification  Battery  and  the  original  ASVAB  used  from  1976 
until  Jctober  1980  had  similar  subtests,  the  Army  aptitude  composites  have 
been  kept  intact  since  1973.  The  aptitude  composites  used  from  i°7l;  until 
ASY  8/9/10  was  implemented,  the  subtests  in  each,  and  ih<=  type  of  s it  i  _  1 
specialties  for  which  the  composites  serve  as  prerequisites  are  siiovr  in 
Table  2. 

With  the  changes  in  ;  lbtests  in  ASVAB  8/9/10,  as  cot  pared  to  the  pre¬ 
vious  version,  the  subtes-:«  in  the  composites  needed  to  be  changed;  a 
related  question  was  whether  the  number  of  composites  should  also  be 
changed . 


^ Ma i e r ,  M.  H.  and  Fuchs,  E.  F.,  Development  and  Evaluation  of  a  New  ACB 
and  Aptitude  Area  System,  Technical  ResearcV:  Note.  239.  Alexandria, 
Virginia:  US  Army  Research  Institute,  September  1972. 


Table  1 


SURTERTS  IN  ASVAB 


SURTFST 

Tests 

in  ASVR  Form 

DESCRIPTION 

6/7 

8/9/10 

Word  Knowledge 

Yes 

Yes 

Understanding  the  meaning 
of  words. 

Ar i thmet ic 

Reasonme 

Yes 

Yes 

Word  problems  that  emphasize 
reasoning  rather  than  mathe¬ 
matical  knowledge. 

Space  Perception 

Yes 

No 

Identifying  a  three-dimen¬ 
sional  figure  obtained  from 
folding  a  flat  pattern. 

Numer ica 1 

Ooerat ions 

Yes 

Yes 

A  speed  test  of  the  four 
arithmetic  operatons  - 
addition,  subtraction, 
multiplication,  division. 

Paragraph 

Comprehension 

No 

Yes 

Understanding  the  meaning  of 
paragraphs. 

Electronics 

Tnformat ion 

Yes 

Yes 

Knowledges  of  electricity, 
radio  principles,  and 
electronics  . 

Mechanical 

Comorehens ion 

Yes 

Yes 

Understanding  of  mechanical 
principles,  such  as  gears, 
pulleys,  and  hydraulics. 

General  Science 

Yes 

Yes 

Knowledge  of  physical  and 
biological  sciences. 

Auto/Shop 

Information  (separate 
subtests  in  ASVAR  6/7) 

Yes 

Yes 

Knowledge  of  automobiles,  shop 
practices  and  use  of  tools. 

Mathematics  Knowledge 

Yes 

Yes 

Knowledges  and  skills  in 
algebra,  geometry  and  fractions 

Coding  Speed 

No 

Yes 

A  speeded  test  to  match 
words  and  numbers. 

Attent ion-to- 
Petai 1 

Yes 

No 

A  speeded  test  to  count  the 
number  of  "C"s  embedded  in 
a  series  of  "0"s. 

Genera  1 

Tnformat ion 

Yes 

No 

Information  on  geography, 
sports,  history,  automobiles. 

Class i f icat ion 

I nventnry 

Yes 

No 

Experience  in  and  preference 

for  activities  related  to 
mechanical,  electronics, 
clerical /admin  is t rat ive , 
and  mascu  1  ine /out door  pursuits. 
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Because  of  the  complexity  of  changing  the  relationship  between  aptitude 
composites  and  individual  MOS,  the  decision  was  made  to  change  onlv  the  sub¬ 
tests  in  each  composite  and  net  to  change  the  number  of  composites.  A  change 
to  the  relationship  between  composites  and  job  specialties  would  require 
'oordination  among  many  Army  agencies,  including  the  Deputy  Chief  of  Jr. iff 
•  r  Personnel,  the  Training  and  Doctrine  Command,  the  Military  Personnel 
C  ter,  and  the  Recruiting  Command,  plus  the  Military  Enlistment  Processing 
Command .  In  addition  several  regulations  and  pamphlets,  such  as  the  manual 
doeribing  the  job  specialties,  would  have  required  extensive  revision.  The 
relationship  of  aptitude  composite  to  ski i 1  specialties  can  he  restructured 
after  the  new  forms  have  been  validated  against  the  modern  job  training  pro¬ 
grams  and  measures  of  job  proficiency.  This  validation  and  restrnctur ir. 
will  require  several  calendar  years  to  complete. 

C r iterion  Measures  of  Successful  Performance . 

Two  criterion  measures,  success  in  skill  training  cour-os  and  scored  rests 
to  measure  job  proficiency,  were  used  for  developing  aptitude  "ompos:  3  v  -s. 

Skill  Training.  Performance  in  skill  training  courses  has  been  the  tradi¬ 
tional  criterion  for  validating  aptitude  tests.  The  pass-fail  scores  used  to 
report  success  in  modern  performance-based  training  courses,  h.iwevc-r,  arc-  not 
satisfactory  criterion  measures  from  a  psychometric  pc-int  of  view.  From  t 
point  of  view  of  how  training  relates  to  job  requirements,  the  new  training 
programs  are  generally  superior  to  the  traditional  courses,  and  the  measure¬ 
ment  problems  arise  largely  because  of  the  particular  way  performance  is 
reported:  pass  or  fail,  with  no  gradations  in  either  score  category. 

Because  students  in  most  of  the  Army  training  courses  are  graded  only  as 
pass-fall,  the  true  relationship  between  aptitude  composite,  scores  and 
success  in  training  is  not  accurately  indicated  by  the  statistical  correlation 
coefficient  traditionally  used  to  report  validity.  The  training  performance 
data  used  in  this  study  came  from  courses  still  using  continuous  final  course 
grades  in  1976-1977. 

Job  Proficiency.  The  second  criterion  for  developing  aptitude  composites  was 
a  measure  of  job  proficiency,  obtained  from  Skill  Qualification  Tests  (S.Tr-  . 
SQTs  are  designed  to  assess  performance  of  critical  job  tasks.  They  are 
criterion  referenced  In  the  sense  that  test  content  is  based  explicitly  on 
job  requirements  and  the  meaning  of  the  test  scores  is  established  by  expert 

judgment  prior  to  administration  of  the  test  rather  than  on  the  basis  of 

score  distributions  obtained  from  administration..  The  content  of  SQTs  is 
a  carefully  selected  sample  from  the  domain  of  critical  tasks  in  a 
specialty.  Tasks  are  selected  because  they  are  especially  critical, 
such  as  a  particular  weapon  system,  or  berause  there  is  a  known  training 
deficiency.  The  focus  on  training  deficiencies  means  that  relatively 
few  on  the  job  can  perform  the  tasks,  and  the  pass  rate  for  these  tasks 
therefore  is  expected  to  be  low.  Since  only  critical  tasks  In  a  specialty 

are  included  in  SQTs,  and  then  only  the  more  difficult  tasks  tend  to  bo 

selected  for  testing,  a  reasonable  inference  is  that  performance  on  the 
SQTs  should  be  a  useful  indicator  of  proficiency  on  the  entire  domain  of 
critical  tasks  in  the  specialty;  that  is,  workers  who  are  proficient  on 
tasks  included  in  an  SQT  are  also  proficient  on  other  t  ».sks  in  the 
specialty.  The  list  of  tasks  In  the  SQT  and  the  measure  themselves 


are  carefully  reviewed  by  job  experts  and  tried  out  on  samples  of  repre¬ 
sentative  job  incumbents  prior  to  operational  administration.  The  process 
of  developing  SQTs  may  be  characterized  as  follows: 

1.  Identify  tasks  for  testing. 

2.  Identify  behaviors  or  steps  essential  for  performing  each  task. 

3.  Develop  measures  to  cover  essential  behaviors,  and  have  these 
measures  reviewed  by  job  experts. 

4.  Tryout  the  measures  on  representative  workers  to  verify  accuracy 
of  measurement;  i.e.,  make  sure  that  measures  discriminate  between  task 
performers  and  nonperformers. 

After  each  step,  the  products  are  reviewed  for  content  validity.  The 
test  content  cannot  be  changed  after  step  3,  when  the  measures  are  approved  by 
experts.  The  tryout  of  step  4  can  be  used  only  to  improve  the  measures,  and 
not  to  change  content.  When  the  development  process  is  followed,  the  validity 
of  the  SQTs  as  measures  of  job  proficiency  is  assured  by  job  experts  and 
representative  workers.  Complete  procedures  for  developing  SQTs  are  contained 
in  a  handbook^,  and  a  more  thorough  discussion  of  the  rationale  and  potential 
uses  of  SQTs  is  presented  in  an  ARI  report. ^ 

PROCEDURES 


Criterion  scores  of  success  in  training  and  of  job  proficiency  were 
obtained  for  each  skill  specialty  that  had  enough  cases  to  permit  statistical 
analysis;  the  minimum  number  to  constitute  a  separate  sample  was  set  at  about 
100  soldiers  in  an  MOS.  A  separate  analysis  was  conducted  for  each  sample. 

The  predictor  measures  were  the  ASVAB  6/7  subtests  that  had  parallel 
counterparts  in  ASVAB  8/9/10.  The  subtests  are  listed  in  Table  1.  In  the 
rase  of  Automotive  and  Shop  Information,  which  were  merged  in  ASVAB  8/9/10, 
the  validity  for  both  subtests  was  computed,  and  Automotive  Information  was 
arbitrarily  chosen  as  the  subtest  to  use  in  test  selection  for  developing  new 
composites.  The  ASVAB  6/7  scores  of  record  were  obtained  from  the  Military 
Enlistment  Pi ocessing  Command  for  Army  recruits  tested  in  early  calendar  year 
i97t,  shortly  after  ASVAB  6/7  was  implemented.  Subtest  raw  scores  were  used 
i.n  the  statistical  analysis. 


^Osborn,  W.  C.(  Campbell,  R.  C.,  Fcrd,  J.  P. ,  Rirshfeld,  S.  F. ,  and  Maler, 

M.  H.  Handbook  for  the  Development  of  Skill  Qualif ic.a1  ton  Te'is.  Technical 
Report  P-77-5.  A1  .cria,  Virginia:  Army  Researcu  Institute,  November 
197  7. 

Viaier,  M.  ii-  and  Rirshfeld,  S.  F.  Criterion  Referenced  Testing;  A 
Large  Scale  Application.  Technical  Research  Report  1193,  Alexandria, 
Virginia:  Army  Research  Institute,  February  1978. 


The  statistical  analysis  consisted  of  computing  the  correlation  between 
each  ASVAB  subtest  and  the  criterion  measure  available  for  each  MOS  sample; 
these  correlations  are  termed  validity  coefficients.  Because  the  cases  in 
each  sample  had  been  selected  at  the  time  of  enlistment  on  the  basis  of 
aptitude  composites,  the  correlation  coefficients  were  corrected  for 
restriction  in  range  to  estimate  the  degree  of  relationship  for  the  full 
tinge  of  ability.  Mean  subtest  validity  coefficients  were  computed  for  all 
samples  that  had  the  same  aptitude  composite  as  a  prerequisite  for  assignment 
into  skill  training.  Using  the  mean  corrected  validity  coefficients,  subtests 
with  the  highest  unique  validity  were  selected  for  each  composite.  As  a  rule, 
subtests  were  added  to  the  composites  as  long  as  they  increased  predictive 
validity.  Exceptions  are  as  follows:  Because  criterion  data  were  not  available 
before  implementing  ASVAB  8/9/10  two  composites,  Electronics  Repair  (EL)  and 
General  Maintenance  (GM),  were  developed  by  expert  judgment.  In  addition,  two 
composites,  Electronics  Repair  (EL)  and  Clerical  (CL),  were  constructed  to  be 
identical  for  all  services.  The  aptitude  composites,  then,  are  based  on  the 
prediction  of  training  success  or  job  proficiency,  or  both,  or  by  expert 
judgment,  or  forced  to  be  identical  for  all  services. 

SQT  scores  subsequently  became  available  for  Electronics  Repair  (EL)  and 
General  Maintenance  (GM)  samples,  and  analyses  were  conducted  to  evaluate  the 
effectiveness  of  the  composites  determined  by  expert  judgment  . 

RESULTS 


The  MOS  samples  used  to  validate  the  ASVAB  subtests  are  listed  in  Table 
3,  together  with  the  type  of  criterion  measure  (training  success  or  SQT ) ^ . 

SQT  scores  were  included  in  the  validation  of  all  composites  except 
Surveillance/Communications  (SC),  for  which  training  success  was  the  sole 
criterion  measure.  Both  types  of  criterion  measures  were  used  for  the  Clerical 
(CL),  Mechanical  Maintenance  (MM),  and  Skilled  Technical  (ST)  composites.  The 
samples  sizes  ranged  from  about  100  cases  to  over  2000;  the  number  of  MOS 
samples  included  for  each  composite  ranged  from  two  for  Electronic  Repair  (EL) 
and  General  Maintenance  (GM)  to  10  for  Clerical  (CL). 

The  correlation  coefficients  between  each  ASVAB  subtest  and  the 
criterion  measures  corrected  for  restriction  in  range,  are  shown  in  Appendix  A. 
The  standard  ASVAB  intercorrelation  matrix  is  shown  in  Appendix  B.  The 
coefficients  for  MOS  that  have  the  same  aptitude  composite  as  a  prerequisite 
were  averaged  to  obtain  a  vector  of  mean  validity  coefficients,  and  the 
mean  validity  vectors  are  shown  in  Table  4. 

The  vectors  of  mean  validity  coefficients  were  used  to  select  the  most 
valid  subtests  to  include  in  each  composite  (except  as  noted  in  the  Procedures 
section  above).  The  test  selection  procedures  first  selected  the  most  valid 


^Validation  of  the  ASVAB  against  success  in  training  was  completed  by  Mr. 
R.  Ross  and  Dr.  M.  Fischl,  ARI.  They  kindly  made  the  results  of  their 
analysis  available  for  use  in  developing  composites  for  ASVAB  8/9/10. 


6 


Table  3 


MOS  SAMPLES  USED  TO  VALIDATE  THE  ASVAB 


Aptitude 

Job  Proficiency 

Training 

Success 

Composite 

SQT 

N 

MOS 

N 

Combat  (CO) 

1 IB  7  7a 

2733 

None 

11B78 

442 

1 1C  7  7 

919 

11C78 

93 

11D77 

356 

11E77 

860 

Field  Artillery  (FA) 

13B78 

436 

None 

13E78 

98 

Electronics 

Repair (EL)b 

24M79 

202 

None 

27E79 

147 

36C79 

2328 

36K79 

2217 

Operators  and 

Food  (OF) 

16D77 

134 

None 

16E77 

93 

16P77 

405 

16R77 

182 

Surveillance  and 

Communication  (SC) 

None 

05B/C 

78 

05  E 

119 

05F 

257 

72E 

233 

Mechanical 

Maintenance  (MM) 

63B78 

131 

62B 

128 

63B 

124 

General 

Maintenance  (GM) 

55B79 

659 

None 

57H79 

220 

Clerical  (CL) 

76D77 

421 

71B 

311 

76P77 

87 

73C 

77 

76Y77 

322 

75(CMF) 

192 

76Y 

124 

Skilled  Technical  (ST) 

95B77 

1013 

91B 

311 

95B 

166 

aFirst  three  characters  are  MOS  and  Last  two  digits  are  year  of  testing. 
bOnly  24M79  and  27E79  used  to  verify  composite. 
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subtest;  then  removed  the  effects  of  that  subtest  from  the  remaining  subtest 
and  criterion  scores;  the  subtest  with  the  next  highest  validity  (after  the 
effects  of  the  first  subtest  had  been  removed)  was  then  selected.  This  process 
was  repeated  until  none  of  the  remaining  subtests  contributed  substantially  to 
the  predictive  validity  of  the  composite.  The  subtests  in  each  composite  are 
shown  in  Table  5. 

There  are  several  exceptions  to  the  above  procedure  for  selecting  subtests 
for  each  composite.  One  exception  is  that  the  General  Technical  (GT)  composite 
traditionally  has  been  defined  as  Word  Knowledge  plus  Arithmetic  Reasoning. 

Since  it  is  not  used  as  a  prerequisite  for  MOS  training,  the  decision  was  made 
to  retain  the  traditional  definition,  but  to  expand  Word  Knowledge  to  the 
Verbal  score  by  also  including  the  Paragraph  Comprehension  subtest.  A  second 
exception  is  the  Clerical  (CL)  composite.  The  data  indicated  that  the 
inclusion  of  the  Arithmetic  Reasoning  subtest  would  have  resulted  in  a 
higher  validity  than  using  the  Verbal,  Coding  Speed,  and  Numerical  Operations 
subtests  by  themselves.  Inclusion  of  Arithmetic  Reasoning  would  have  increased 
the  validity  of  the  Clerical  (CL)  composite  by  three  points,  from  .55  to  .58 
Since  by  excluding  Arithmetic  Reasoning,  the  Clerical  (CL)  composite  is  the 
same  for  all  services,  the  decision  was  to  strive  for  communality  among  all 
services  and  delete  Arithmetic  Reasoning. 

Another  exception  is  the  computation  of  Electronic  Repair  (EL)  and 
General  Maintenance  (GM)  aptitude  composites.  The  SQT  data  for  these 
composites  did  not  become  available  for  analysis  until  fall  1980,  long  after 
the  subtests  in  each  composite  had  to  be  specified.  Because  ASVAB  8/9/10  was 
scheduled  for  implementation  on  1  October  1980,  the  composites  had  to  be 
defined  by  fall  1979  to  allow  for  printing  of  the  testing  and  scoring  materials. 
The  results  for  these  two  composites  are  included  here  to  complete  the  validity 
picture  for  the  ASVAB. 

For  Electronics  Repair  (EL)  only  two  skill  specialties  were  used  to  verify 
the  validity  of  the  composite,  which  had  been  defined  by  expert  judgment.  These 
are  highly  technical  specialties  (24M  and  27E)  typical  of  the  skills  and 
knowledge  required  of  other  electronics  repair  jobs.  The  SQT  for  two  other 
electronics  specialties,  36C  and  36K,  were  also  available  but  were  used  only 
incidentally.  One  reason  Is  that  the  skill  and  knowledge  requirements  for  these 
specialties  are  considerably  lower  than  the  highly  technical  repair  MOS.  The 
second  is  that  the  SQT  for  36C  and  36K  are  suspect.  For  both  MOS,  about  a 
quarter  of  the  sample  received  perfect  SQT  scores  which  suggests  that  the 
tests  were  inordinately  easy.  Therefore  only  the  highly  technical  specialties 
(24M  and  27E)  were  used  for  verifying  the  Electronics  Repair  (EL)  composite. 

The  test  selection  procedure  to  identify  the  most  valid  subtests  in  the 
Electonics  Repair  (EL)  composite  resulted  in  selecting  Electronics  Information 
and  Mathematics  Knowledge  as  the  first  two  subtests  chosen,  and  both  of  these 
are  in  the  composite.  The  other  two  subtests  in  Electronics  Repair  (EL), 
Arithmetic  Reasoning  and  General  Science,  however,  were  not  selected  until 
other  subtests  had  been  selected.  The  best  combination  of  subtests  included 
Electronics  Information  and  Mathematics  Knowledge,  plus  Auto/Shop  Information 
and  Mechanical  Comprehension  rather  than  Arithmetic  Reasoning  and  General 
Science;  the  best  set  of  subtests  had  a  validity  coefficient  of  .62,  as 
compared  to  .59  for  the  operational  set  of  subtests. 
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The  General  Maintenance  (GM)  composite,  which  had  also  been  defined  on  the 
basis  of  expert  judgment,  was  verified  by  the  results  for  the  two  skill 
specialties  that  had  SQT  scores  available.  This  composite  in  ASVAB  8/9/10 
contains  Mathematics  Knowledge,  Electronics  Information,  General  Science,  and 
Auto/Shop  Information.  For  both  specialties.  Mathematics  Knowledge  and 
Electronics  Information  were  the  first  two  tests  selected,  and  for  the  55B  MOS 
the  third  and  fourth  test  were  General  Science  and  Auto/Shop  Information.  A 
similar  result  was  obtained  for  the  57H  MOS,  except  that  Numerical  Operations 
was  selected  as  the  fourth  subtest  with  General  Science  as  the  fifth.  Numer¬ 
ical  Operation  is  a  speeded  test  of  perceptual  accuracy,  and  has  no  apparent 
relationship  to  job  requirements  in  the  General  Maintenance  (GM)  area.  There¬ 
fore  it  would  not  be  included  in  the  composite  on  content  considerations. 

These  results  confirmed  the  General  Maintenance  (GM)  composite. 

One  of  the  significant  improvements  is  the  new  Operators/Food  (OF) 
composite.  The  old  Operators/Food  (OF)  composite  contained  the  General  Infor¬ 
mation,  Attention  to  Detail,  and  Attentiveness  subtests.  Only  General  Information 
required  the  ability  to  demonstrate  verbal  skills  and  knowledge;  the  Attentive¬ 
ness  score  measures  interest  and  experiences,  and  Attention  to  Detail  is  a 
measure  of  perceptual  speed.  The  new  Operators/Food  (OF)  composite  contains 
the  Verbal,  Mechanical  Comprehension  and  Automotive -Shop  Information  subtests, 
plus  Numerical  Operations,  a  measure  of  perceptual  speed. 

The  new  composites  were  found  to  have  adequate  validity  to  serve  as 
effective  predictors  of  job  proficiency.  The  validity  coefficients  for  pre¬ 
dicting  SQT  scores  are  comparable  to  those  typically  found  for  predicting 
final  course  grades  in  traditional  training  programs.  Thus  the  ASVAB  is  an 
effective  predictor  of  the  skills  and  knowledges  required  to  perform  the 
variety  of  Army  skills,  as  measured  by  Skill  Qualification  Tests. 


DISCUSSION  AND  CONCLUSIONS 

Criterion  Measures  in  Modern  Training  Programs 

The  empirical  validity  of  the  ASVAB  will  become  increasingly  difficult  to 
demonstrate,  if  indeed  the  battery  is  still  valid,  for  training  programs 
employing  modern  instructional  technology.  Army  training  programs  have  been 
redesigned  during  the  1970's  to  have  the  following  characteristics  that 
impinge  on  the  validity  of  the  ASVAB: 

a.  pass-fail  scoring,  with  no  rank  order  information  reported  within  each 
score  category; 

b.  performance -based  training  and  testing,  with  focus  on  developing  and 
evaluating  the  skills  and  knowledges  required  to  perform  prescribed  job  tasks; 

c.  self -pacing,  with  each  student  terminating  training  in  each  unit  of 
instruction  as  soon  as  the  minimum  requirements  in  that  unit  are  satisfied. 
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Traditional  training  programs,  in  contrast,  featured  training  content, 
techniques,  and  evaluation  procedures  that  maximized  the  relationship  to  paper- 
and-pencil  aptitude  tests.  Course  content  tended  to  focus  on  principles, 
knowledges  and  functions  as  presented  in  lectures  and  demonstrations  by  the 
instructor,  with  occasional  practical  •;  Lercises  to  provide  hands-on  training 
for  specific  job  tasks.  All  students  in  the  traditional  courses  were  exposed 
to  the  same  material  for  the  same  amount  of  time  (group  paced)  instead  of 
moving  on  as  soon  as  the  minimal  requirements  in  an  instructional  unit  were 
met.  The  practice  of  exposing  all  students  to  the  same  material  for  the  same 
amount  of  time  tended  to  produce  variation  in  the  levels  of  achievement.  And 
perhaps  most  important,  in  traditional  courses  final  course  grades  were 
determined  primarily  by  paper-and -pencil  achievement  tests  that  bear  a  close 
resemblance  to  the  aptitude  tests.  In  some  respects  the  high  validity  of 
aptitude  tests  for  predicting  final  course  grades  in  traditional  training  pro¬ 
rams  may  have  been  an  artifact  of  the  way  achievement  was  evaluated. 

Job  Proficiency  as  a  Criterion 

The  high  degree  of  relationship  between  ASVAB  and  SQT  scores  came  as  a 
surprise  to  many  observers.  Some  of  the  reasons  have  been  alluded  to  in  this 
report,  and  the  concluding  section  may  be  a  good  place  to  address  them  more 
spec! fically . 

Proficiency  vs  Performance.  A  long  standing  argument  is  that  aptitude 
tests  cannot  predict  job  performance,  and  ASVAB  scores  therefore  should  not 
be  related  to  measures  of  job  performance.  Performance,  which  is  what  a  person 
actually  does  on  the  job,  is  a  different  concept  than  proficiency,  which 
refers  to  the  skills  and  knowledges  a  person  possesses.  Performance  includes 
components  of  motivation  and  supervision,  in  addition  to  proficiency.  SQTs 
are  measures  of  proficiency  rather  than  performance,  and  therefore  a  more 
precise  statement  is  that  ASVAB  predicts  job  proficiency  rather  than  job 
performance. 

Quality  of  ASVAB.  The  original  ASVAB  fell  on  hard  times  during  its  life¬ 
time  from  1976  until  October  1980,  when  new  forms  were  introduced.  Scores 
of  the  earlier  version  were  suspect  because  of  wide-spread  test  compromise 
(coaching  on  the  test)  as  reported  in  the  nation's  press  in  late  1977  and 
an  Inflated  score  scale  which  overestimated  the  aptitude  of  recruits.  The 
scores,  therefore  were  untrustworthy,  and  a  natural  inference  is  that  they 
could  no'  predict  subsequent  performance. 

Quality  of  the  SQTs  as  measures  of  job  proficiency.  A  final  concern  is 
whether  SQTs  are  reliable  and  valid  of  job  proficiency.  The  meaning  of  SQT 
scores  lias  been  controversial  in  the  Army,  and  SQTs  themselves  have  been 
criticized  in  many  quarters  as  poor  measures  of  performance.  One  reason  may 
be  that  many  examinees  "fail"  the  test;  and  therefore  in  many  skill 
specialties  the  majority  of  the  job  incumbents  are  nominally  unqualified. 

Since  this  outcome  is  clearly  undesirable,  a  conclusion  is  that  the  test 
themselves  are  not  functioning  properly. 
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The  passing  score  for  SQTs  is  an  arbitrarily  chosen  namber;  currently  60 
percent  of  the  items  must  be  correct  to  pass  the  test.^  The  difficulty  of 
the  test  is  not  taken  into  account  in  setting  the  passing  score.  The  percent 
passing  an  SQT  is  a  joint  function  of  the  proficiency  of  the  examinees  and 
difficulty  of  the  tasks  included  in  an  SQT.  It  may  be  recalled  that  a  major 
consideration  in  selecting  tasks  for  SQTs  is  known  training  deficiencies, 
which  means  that  these  tasks  tend  to  have  low  pass  rates.  The  passing  score 
of  60  percent  may  be  unduly  high  for  specialties  with  many  training  deficiencies. 
However,  because  the  policy  is  to  set  the  passing  score  prior  to  test 
administration  and  to  have  all  passing  scores  the  same,  the  pass  rates  for 
specialties  will  continue  to  fluctuate  until  test  developers  lea^n  how  to 
pitch  the  test  at  the  desired  level  of  difficulty. 

A  key  element  in  assessing  the  quality  of  SQTs  as  measures  of  job  profi¬ 
ciency  is  to  examine  the  developmental  process.  Briefly  reviewed,  all  the 
tasks  selected  for  testing  are  critical  to  the  job  and  many  of  them  may  be 
especially  difficult.  The  tasks  and  measures  are  reviewed  by  job  experts  and 
tried  out  on  representative  workers.  .'Examinees  are  informed  beforehand  about 
the  tasks  that  are  covered  by  the  SQI,  which  gives  than  an  equal  opportunity 
to  prepare.  The  process  ts  designed  to  produce  quality  measures  of  proficiency, 
and  there  is  no  reason  to  question  the  generalizability  of  scores  on  the  tasks 
in  the  test  to  the  ->ther  critical  tasks  in  a  specialty.  It  is  possible  that 
some  SQTs  are  not  velopcd  according  to  this  model,  and  the  quality  may  be 
lower.  Individual  ‘.receptions  do  not  negate  the  quality  of  the  procedures,  and 
therefore  the  test  Toped  according  to  the  procedures  are  expected  to  be 
adequate  criterion  measures  of  job  proficiency. 

The  proficiency  measures  used  in  this  validation  effort  were  the  first 
generation  oc  SQTs,  which  contained  a  large  component  of  paper -and -pencil  type 
achievement  tests.  The  second  generation  SQT  beginning  in  1980’ s  emphasizes 
hand -on  performance  more  than  paper-and -pencil  tests  for  some  specialties; 
the  validity  of  the  ASVAB  for  these  SQTs  is  not  know  at  the  time  of  this 
writing. 

The  changing  nature  of  the  criterion  measures  in  the  Army  will  require  new 
strategies  for  validating  ASVAB.  Traditionally,  the  ASVAB  has  bee  ;  described  as 
a  measure  of  trainablli ty ,  which  generally  meant  the  ability  to  predict  success 
in  the  traditional  training  programs.  Modern  instructional  technology,  with 
pass-fail  scoring  of  performance,  is  forcing  a  reevaluation  of  the  utility  of 
ASVAB  as  a  predictor  of  training  success.  The  interest  Is  growing  among 
personnel  managers  in  the  effectiveness  of  ASVAB  as  a  predictor  of  job 
performance . 


^Technically  speaking  60  percent  of  the  scorable  units  must  be  scored  as  "Go," 
where  a  scorable  unit  is  a  set  of  written  items  or  performance  measures 
related  to  a  specific  task.  The  number  of  items  correct  need  not  be  60 
percent  in  order  to  be  "Go"  on  60  percent  of  the  scorable  units;  but  the 
relationship  is  close  enough  for  these  purposes,  and  it  is  easier  to  under¬ 
stand  that  60  percent  of  the  items  must  be  correct. 
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SQTs  promise  to  provide  a  breakthrough  in  providing  operational  measures  of 
iob  proficiency,  and  they  proved  in  this  research  effort  to  be  useful  criterion 
measures.  Extensive  effort  will  be  required  to  develop  a  research  program  to 
validate  the  ASVAR  against  measures  of  success  in  modern  skill  training  courses, 
and  against  iob  performance,  the  criterion  so  long  sought  by  the  testing 
community. 
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